{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Deep Learning with TensorFlow\n",
    "\n",
    "In this notebook, we will go over the basics of Deep Learning using TensorFlow 2. TensorFlow is a framework which allows manipulations of tensors using constructs similar to numpy. In addition it has modules to allow auto differentiation to carryout back-propagation which forms the backbone of training a neural network.\n",
    "\n",
    "Prior to version 2, TensorFlow had two phases, first phase to build graph of computation and then in 2nd phase using `session` to pass data through the grapah and run back propagation. However with the introduction of eager execution in Tensorflow 2.X, it now has simplified a lot and the differences between PyTorch and TensorFlow have become very narrow. \n",
    "\n",
    "Ofcourse for production setup, you could still use the pre-complied graph to speed up things.\n",
    "\n",
    "## What are Neural Networks\n",
    "\n",
    "Deep learning is based on Artificial Neural networks which are made up of neurons. A neuron takes inputs, calculates the weighted sum and then passes the sum through some kind of non-linear function (called activation function) as shown below:\n",
    "\n",
    "![Neuron](./images/neuron.png \"Neuron\")\n",
    "\n",
    "<br/>\n",
    "<br/>\n",
    "\n",
    "We stack these neurons to make a neural network as shown below:\n",
    "![Neuron](./images/nn.svg \"Neuron\")\n",
    "\n",
    "Let us now create a neural network in PyTorch. We will use this network to train a model to take MNIST data as input and produce the class it belongs to.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-10-16 19:18:34.805866: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
      "2023-10-16 19:18:34.853131: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
      "2023-10-16 19:18:34.853181: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
      "2023-10-16 19:18:34.853203: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
      "2023-10-16 19:18:34.862346: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
      "To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2023-10-16 19:18:35.716915: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MNIST \n",
    "\n",
    "MNIST dataset has images 28x28 pixels  = 784 pixels. \n",
    "\n",
    "We will have 10 units at the output layer to signify the digit (0-9) the image belongs to.\n",
    "\n",
    "Let us first load the data and print some images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "x_train, x_test = x_train / 255.0, x_test / 255.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 28, 28)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "image size of `(60000,28,28)` means that we have 60000 images in the x_train dataset , with each image of size (28x28) (height x width)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f24670be800>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbIklEQVR4nO3df2xV9f3H8dct0Atqe7HU9vbyywIqRgQjSm1UpqOh7RYDygyoS9AZDa64KVO3blrUzdSxZDMa1CUzdEYRZRkQyUam1ZZsa3GghBi1o02ROmhRlt4LxRbSfr5/8PWOKy14Lvf2fe/l+UhOQu89H857Zxeent7Lqc855wQAwDDLsh4AAHB2IkAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMDESOsBvm5gYED79u1TTk6OfD6f9TgAAI+cczp06JBCoZCysoa+zkm5AO3bt08TJ060HgMAcIY6Ojo0YcKEIZ9PuW/B5eTkWI8AAEiA0/19nrQArV69WhdeeKFGjx6tkpISvffee99oHd92A4DMcLq/z5MSoNdff10rVqzQypUr9f7772vWrFkqLy/XgQMHknE4AEA6ckkwZ84cV1VVFf26v7/fhUIhV1tbe9q14XDYSWJjY2NjS/MtHA6f8u/7hF8BHT16VDt27FBZWVn0saysLJWVlampqemk/fv6+hSJRGI2AEDmS3iAvvjiC/X396uwsDDm8cLCQnV2dp60f21trQKBQHTjE3AAcHYw/xRcdXW1wuFwdOvo6LAeCQAwDBL+74Dy8/M1YsQIdXV1xTze1dWlYDB40v5+v19+vz/RYwAAUlzCr4Cys7M1e/Zs1dfXRx8bGBhQfX29SktLE304AECaSsqdEFasWKGlS5fqqquu0pw5c/TMM8+op6dHd911VzIOBwBIQ0kJ0OLFi/X555+rpqZGnZ2duuKKK7Rly5aTPpgAADh7+ZxzznqIE0UiEQUCAesxAABnKBwOKzc3d8jnzT8FBwA4OxEgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATI60HAJJh9uzZca1bsmSJ5zU/+MEPPK/56KOPPK9pbm72vCZev/jFLzyvOXr0aBImQSbjCggAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMOFzzjnrIU4UiUQUCASsx0AKiefGmDU1NXEdKzs7O651mebWW2/1vOZPf/pTEiZBOguHw8rNzR3yea6AAAAmCBAAwETCA/T444/L5/PFbNOnT0/0YQAAaS4pP5Dusssu09tvv/2/g4zk594BAGIlpQwjR45UMBhMxm8NAMgQSXkPaPfu3QqFQpoyZYruuOMO7d27d8h9+/r6FIlEYjYAQOZLeIBKSkpUV1enLVu26IUXXlB7e7uuv/56HTp0aND9a2trFQgEotvEiRMTPRIAIAUlPECVlZW69dZbNXPmTJWXl+svf/mLuru79cYbbwy6f3V1tcLhcHTr6OhI9EgAgBSU9E8HjB07VhdffLFaW1sHfd7v98vv9yd7DABAikn6vwM6fPiw2traVFRUlOxDAQDSSMID9NBDD6mxsVF79uzRP//5T918880aMWKEbrvttkQfCgCQxhL+LbjPPvtMt912mw4ePKgLLrhA1113nZqbm3XBBRck+lAAgDTGzUiR8vLz8z2v2bNnT1zHOvfcc+Nal2l6e3s9r7nrrrs8r1m3bp3nNUgf3IwUAJCSCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATSf+BdMCZ+uKLLzyvefrpp+M6Vk1Njec1o0aN8rwmHA57XjOcN+kdPXq05zULFizwvIabkZ7duAICAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACZ9zzlkPcaJIJDKsd/0FTvSf//zH85pQKOR5TWdnp+c1wWDQ85rhdOmll3pe88knnyRhEqSKcDis3NzcIZ/nCggAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMDHSegAglTzxxBOe19TU1HheM378eM9rUt2YMWOsR0Ca4QoIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDhc8456yFOFIlEFAgErMcAvrF4biz6r3/9y/OaoqIiz2uG07Zt2zyvueaaa5IwCVJFOBxWbm7ukM9zBQQAMEGAAAAmPAdo69atuummmxQKheTz+bRx48aY551zqqmpUVFRkcaMGaOysjLt3r07UfMCADKE5wD19PRo1qxZWr169aDPr1q1Ss8++6xefPFFbdu2Teeee67Ky8vV29t7xsMCADKH55+IWllZqcrKykGfc87pmWee0aOPPqoFCxZIkl5++WUVFhZq48aNWrJkyZlNCwDIGAl9D6i9vV2dnZ0qKyuLPhYIBFRSUqKmpqZB1/T19SkSicRsAIDMl9AAdXZ2SpIKCwtjHi8sLIw+93W1tbUKBALRbeLEiYkcCQCQosw/BVddXa1wOBzdOjo6rEcCAAyDhAYoGAxKkrq6umIe7+rqij73dX6/X7m5uTEbACDzJTRAxcXFCgaDqq+vjz4WiUS0bds2lZaWJvJQAIA05/lTcIcPH1Zra2v06/b2du3cuVN5eXmaNGmSHnjgAf3qV7/SRRddpOLiYj322GMKhUJauHBhIucGAKQ5zwHavn27brzxxujXK1askCQtXbpUdXV1euSRR9TT06N7771X3d3duu6667RlyxaNHj06cVMDANIeNyMFTvCjH/3I85qrrrrK85rvf//7ntf4fD7Pa4bTU0895XnNo48+moRJkCq4GSkAICURIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADAhOcfxwAMtyuuuMLzmr/97W9xHWvcuHGe12Rl8d9xkvTKK69Yj4A0w58cAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAENyNFyps9e7bnNeeff35cx+LGovF76qmnPK9ZtGhREiZBuuBPGwDABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABggpuRIuW99NJLntdMnjw5rmNVV1d7XjNyJH+MJGn8+PHWIyDNcAUEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJjgLorISDU1NXGta2lp8bzm/PPPj+tYXsVz09Onn346rmP5/f641gFecAUEADBBgAAAJjwHaOvWrbrpppsUCoXk8/m0cePGmOfvvPNO+Xy+mK2ioiJR8wIAMoTnAPX09GjWrFlavXr1kPtUVFRo//790e211147oyEBAJnH87ualZWVqqysPOU+fr9fwWAw7qEAAJkvKe8BNTQ0qKCgQJdcconuu+8+HTx4cMh9+/r6FIlEYjYAQOZLeIAqKir08ssvq76+Xr/+9a/V2NioyspK9ff3D7p/bW2tAoFAdJs4cWKiRwIApKCE/zugJUuWRH99+eWXa+bMmZo6daoaGho0b968k/avrq7WihUrol9HIhEiBABngaR/DHvKlCnKz89Xa2vroM/7/X7l5ubGbACAzJf0AH322Wc6ePCgioqKkn0oAEAa8fwtuMOHD8dczbS3t2vnzp3Ky8tTXl6ennjiCS1atEjBYFBtbW165JFHNG3aNJWXlyd0cABAevMcoO3bt+vGG2+Mfv3V+zdLly7VCy+8oF27dumPf/yjuru7FQqFNH/+fP3yl7/k3lIAgBg+55yzHuJEkUhEgUDAegwg5fh8Ps9rnn/++biOtWzZMs9r/vvf/3peM2fOHM9r2traPK+BjXA4fMr39bkXHADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwk/EdyA0iOeH6kSTx3tY5Xf3//sKxB5uAKCABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwwc1IgTTxhz/8wXqEU1qzZo3nNXv27En8IEgbXAEBAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACa4GWmGKSgo8Lzmr3/9a1zHqqur87zmueeei+tYmWbixIme13zve99LwiSJE8/NSHF24woIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDBzUgzzPr16z2vufLKK+M6Vjw31Ozo6PC85t///rfnNR999JHnNZJ0ww03eF4zY8YMz2t+9rOfeV7j9/s9r4nX2rVrPa/59NNPkzAJMhlXQAAAEwQIAGDCU4Bqa2t19dVXKycnRwUFBVq4cKFaWlpi9unt7VVVVZXGjRun8847T4sWLVJXV1dChwYApD9PAWpsbFRVVZWam5v11ltv6dixY5o/f756enqi+zz44IN68803tX79ejU2Nmrfvn265ZZbEj44ACC9efoQwpYtW2K+rqurU0FBgXbs2KG5c+cqHA7rpZde0tq1a/Xtb39b0vGfknjppZequblZ11xzTeImBwCktTN6DygcDkuS8vLyJEk7duzQsWPHVFZWFt1n+vTpmjRpkpqamgb9Pfr6+hSJRGI2AEDmiztAAwMDeuCBB3TttddGP4ba2dmp7OxsjR07NmbfwsJCdXZ2Dvr71NbWKhAIRLd4PtoLAEg/cQeoqqpKH374odatW3dGA1RXVyscDke3eP6dCAAg/cT1D1GXL1+uzZs3a+vWrZowYUL08WAwqKNHj6q7uzvmKqirq0vBYHDQ38vv9w/rP7ADAKQGT1dAzjktX75cGzZs0DvvvKPi4uKY52fPnq1Ro0apvr4++lhLS4v27t2r0tLSxEwMAMgInq6AqqqqtHbtWm3atEk5OTnR93UCgYDGjBmjQCCgu+++WytWrFBeXp5yc3N1//33q7S0lE/AAQBieArQCy+8IOnk+2WtWbNGd955pyTpd7/7nbKysrRo0SL19fWpvLxczz//fEKGBQBkDp9zzlkPcaJIJKJAIGA9RtqqqKjwvGb16tVxHWvKlClxrfOqu7vb85o9e/bEdaxLL73U85rheg8znj+qn3/+eVzHmjp1quc1hw8fjutYyFzhcFi5ublDPs+94AAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCu2FDa9eujWvdxx9/7HnNk08+GdexIH355Zee15xzzjlJmAT4ZrgbNgAgJREgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJkZaDwB7t99+e1zrRo8e7XlNTk5OXMfy6pprrolr3fXXX5/gSQbX29vreU1FRUUSJgHscAUEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJjwOeec9RAnikQiCgQC1mMAAM5QOBxWbm7ukM9zBQQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMeApQbW2trr76auXk5KigoEALFy5US0tLzD433HCDfD5fzLZs2bKEDg0ASH+eAtTY2Kiqqio1Nzfrrbfe0rFjxzR//nz19PTE7HfPPfdo//790W3VqlUJHRoAkP5Getl5y5YtMV/X1dWpoKBAO3bs0Ny5c6OPn3POOQoGg4mZEACQkc7oPaBwOCxJysvLi3n81VdfVX5+vmbMmKHq6modOXJkyN+jr69PkUgkZgMAnAVcnPr7+913v/tdd+2118Y8/vvf/95t2bLF7dq1y73yyitu/Pjx7uabbx7y91m5cqWTxMbGxsaWYVs4HD5lR+IO0LJly9zkyZNdR0fHKferr693klxra+ugz/f29rpwOBzdOjo6zE8aGxsbG9uZb6cLkKf3gL6yfPlybd68WVu3btWECRNOuW9JSYkkqbW1VVOnTj3peb/fL7/fH88YAIA05ilAzjndf//92rBhgxoaGlRcXHzaNTt37pQkFRUVxTUgACAzeQpQVVWV1q5dq02bNiknJ0ednZ2SpEAgoDFjxqitrU1r167Vd77zHY0bN067du3Sgw8+qLlz52rmzJlJ+R8AAEhTXt730RDf51uzZo1zzrm9e/e6uXPnury8POf3+920adPcww8/fNrvA54oHA6bf9+SjY2Nje3Mt9P93e/7/7CkjEgkokAgYD0GAOAMhcNh5ebmDvk894IDAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJhIuQA556xHAAAkwOn+Pk+5AB06dMh6BABAApzu73OfS7FLjoGBAe3bt085OTny+Xwxz0UiEU2cOFEdHR3Kzc01mtAe5+E4zsNxnIfjOA/HpcJ5cM7p0KFDCoVCysoa+jpn5DDO9I1kZWVpwoQJp9wnNzf3rH6BfYXzcBzn4TjOw3Gch+Osz0MgEDjtPin3LTgAwNmBAAEATKRVgPx+v1auXCm/3289iinOw3Gch+M4D8dxHo5Lp/OQch9CAACcHdLqCggAkDkIEADABAECAJggQAAAE2kToNWrV+vCCy/U6NGjVVJSovfee896pGH3+OOPy+fzxWzTp0+3Hivptm7dqptuukmhUEg+n08bN26Med45p5qaGhUVFWnMmDEqKyvT7t27bYZNotOdhzvvvPOk10dFRYXNsElSW1urq6++Wjk5OSooKNDChQvV0tISs09vb6+qqqo0btw4nXfeeVq0aJG6urqMJk6Ob3IebrjhhpNeD8uWLTOaeHBpEaDXX39dK1as0MqVK/X+++9r1qxZKi8v14EDB6xHG3aXXXaZ9u/fH93+/ve/W4+UdD09PZo1a5ZWr1496POrVq3Ss88+qxdffFHbtm3Tueeeq/LycvX29g7zpMl1uvMgSRUVFTGvj9dee20YJ0y+xsZGVVVVqbm5WW+99ZaOHTum+fPnq6enJ7rPgw8+qDfffFPr169XY2Oj9u3bp1tuucVw6sT7JudBku65556Y18OqVauMJh6CSwNz5sxxVVVV0a/7+/tdKBRytbW1hlMNv5UrV7pZs2ZZj2FKktuwYUP064GBARcMBt1vfvOb6GPd3d3O7/e71157zWDC4fH18+Ccc0uXLnULFiwwmcfKgQMHnCTX2NjonDv+//2oUaPc+vXro/t8/PHHTpJramqyGjPpvn4enHPuW9/6lvvxj39sN9Q3kPJXQEePHtWOHTtUVlYWfSwrK0tlZWVqamoynMzG7t27FQqFNGXKFN1xxx3au3ev9Uim2tvb1dnZGfP6CAQCKikpOStfHw0NDSooKNAll1yi++67TwcPHrQeKanC4bAkKS8vT5K0Y8cOHTt2LOb1MH36dE2aNCmjXw9fPw9fefXVV5Wfn68ZM2aourpaR44csRhvSCl3M9Kv++KLL9Tf36/CwsKYxwsLC/XJJ58YTWWjpKREdXV1uuSSS7R//3498cQTuv766/Xhhx8qJyfHejwTnZ2dkjTo6+Or584WFRUVuuWWW1RcXKy2tjb9/Oc/V2VlpZqamjRixAjr8RJuYGBADzzwgK699lrNmDFD0vHXQ3Z2tsaOHRuzbya/HgY7D5J0++23a/LkyQqFQtq1a5d++tOfqqWlRX/+858Np42V8gHC/1RWVkZ/PXPmTJWUlGjy5Ml64403dPfddxtOhlSwZMmS6K8vv/xyzZw5U1OnTlVDQ4PmzZtnOFlyVFVV6cMPPzwr3gc9laHOw7333hv99eWXX66ioiLNmzdPbW1tmjp16nCPOaiU/xZcfn6+RowYcdKnWLq6uhQMBo2mSg1jx47VxRdfrNbWVutRzHz1GuD1cbIpU6YoPz8/I18fy5cv1+bNm/Xuu+/G/PiWYDCoo0ePqru7O2b/TH09DHUeBlNSUiJJKfV6SPkAZWdna/bs2aqvr48+NjAwoPr6epWWlhpOZu/w4cNqa2tTUVGR9ShmiouLFQwGY14fkUhE27ZtO+tfH5999pkOHjyYUa8P55yWL1+uDRs26J133lFxcXHM87Nnz9aoUaNiXg8tLS3au3dvRr0eTnceBrNz505JSq3Xg/WnIL6JdevWOb/f7+rq6txHH33k7r33Xjd27FjX2dlpPdqw+slPfuIaGhpce3u7+8c//uHKyspcfn6+O3DggPVoSXXo0CH3wQcfuA8++MBJcr/97W/dBx984D799FPnnHNPP/20Gzt2rNu0aZPbtWuXW7BggSsuLnZffvml8eSJdarzcOjQIffQQw+5pqYm197e7t5++2135ZVXuosuusj19vZaj54w9913nwsEAq6hocHt378/uh05ciS6z7Jly9ykSZPcO++847Zv3+5KS0tdaWmp4dSJd7rz0Nra6p588km3fft2197e7jZt2uSmTJni5s6dazx5rLQIkHPOPffcc27SpEkuOzvbzZkzxzU3N1uPNOwWL17sioqKXHZ2ths/frxbvHixa21ttR4r6d59910n6aRt6dKlzrnjH8V+7LHHXGFhofP7/W7evHmupaXFdugkONV5OHLkiJs/f7674IIL3KhRo9zkyZPdPffck3H/kTbY/35Jbs2aNdF9vvzyS/fDH/7QnX/++e6cc85xN998s9u/f7/d0ElwuvOwd+9eN3fuXJeXl+f8fr+bNm2ae/jhh104HLYd/Gv4cQwAABMp/x4QACAzESAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAm/g9GGZle8qVbfQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Let us plot one image\n",
    "plt.imshow(x_train[10].squeeze(), cmap='Greys_r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " flatten (Flatten)           (None, 784)               0         \n",
      "                                                                 \n",
      " dense (Dense)               (None, 192)               150720    \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 128)               24704     \n",
      "                                                                 \n",
      " dense_2 (Dense)             (None, 10)                1290      \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 176714 (690.29 KB)\n",
      "Trainable params: 176714 (690.29 KB)\n",
      "Non-trainable params: 0 (0.00 Byte)\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "  tf.keras.layers.Dense(192, activation='relu'),\n",
    "  tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dense(10)\n",
    "])\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forward pass and Calculate Cross Entropy Loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# function to view the probability of classification of digit\n",
    "def view_classification(img, probs):\n",
    "    fig, (ax1, ax2) = plt.subplots(figsize=(6,7), ncols=2)\n",
    "    ax1.imshow(img)\n",
    "    ax1.axis('off')\n",
    "    ax2.barh(np.arange(10), probs)\n",
    "    ax2.set_aspect(0.1)\n",
    "    ax2.set_yticks(np.arange(10))\n",
    "    ax2.set_yticklabels(np.arange(10).astype(int), size='large');\n",
    "    ax2.set_title('Probability')\n",
    "    ax2.set_xlim(0, 1.1)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAERCAYAAACq8dRTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAmvklEQVR4nO3deVxU9f4/8NewDZszKrih45IKlmgZNxCuWze+gnrB8quCZWBp2qLkQ9voZoRmYHq1bqbpvYZLmJaplNdC6Rdt4m6LW5gJASqZJYMCwzKf3x/3y9xGZg44DpzD4fV8POZR81nOvDqBbz9nmaMRQggQERGRrFzkDkBEREQsyERERIrAgkxERKQALMhEREQKwIJMRESkACzIRERECsCCTEREpAAsyERERArAgkxERKQALMhERDLSaDSYPXu207a3fv16aDQaHD58uNGxo0aNwqhRoyzvCwoKoNFosH79ekvbSy+9BI1G47R8ZB8LMhGRDfWFrf7l6emJwMBAzJ49G6WlpXLHk9Urr7yCnTt3yh1DdViQiYgkLFy4EJs2bcLKlSsRERGB1atXIzw8HBUVFXJHu2l79uzBnj17JMe88MILqKystGpjQW4ebnIHICJSsjFjxuBPf/oTAGDGjBnw8/PD8uXLkZWVhSlTpjQYf+3aNfj4+LR0TId4eHg0OsbNzQ1ubiwVLYErZCKiG/CXv/wFAHDu3DlMmzYNvr6+OHv2LMaOHYt27drhgQceAPCfwjx//nwYDAZotVoEBQVh2bJlsPeAvczMTAQFBcHT0xMhISH44osvrPoLCwvx+OOPIygoCF5eXvDz88OkSZNQUFBgc3sVFRWYNWsW/Pz8oNPpkJCQgN9//91qzPXnkG25/hyyRqPBtWvXsGHDBsvh/GnTpuGzzz6DRqPBjh07Gmxj8+bN0Gg0yMvLk/ysto5/7SEiugFnz54FAPj5+QEAamtrERUVhWHDhmHZsmXw9vaGEAKxsbH47LPPMH36dNxxxx3Izs7G008/jZKSEqxYscJqm59//jm2bt2KpKQkaLVarFq1CtHR0Th48CCCg4MBAIcOHcK+ffsQHx+PHj16oKCgAKtXr8aoUaNw8uRJeHt7W21z9uzZaN++PV566SX88MMPWL16NQoLC5Gbm3tTF2lt2rQJM2bMQGhoKGbOnAkA6Nu3L4YOHQqDwYDMzEzcd999VnMyMzPRt29fhIeHO/y5bYIgIqIGMjIyBACRk5MjLl26JIqKisSWLVuEn5+f8PLyEsXFxSIxMVEAEM8995zV3J07dwoA4uWXX7ZqnzhxotBoNOLHH3+0tAEQAMThw4ctbYWFhcLT01Pcd999lraKiooGGfPy8gQAsXHjxga5Q0JCRHV1taX91VdfFQBEVlaWpW3kyJFi5MiRlvfnzp0TAERGRoalLSUlRVxfKnx8fERiYmKDPMnJyUKr1YorV65Y2n755Rfh5uYmUlJSGownazxkTUQkITIyEp06dYLBYEB8fDx8fX2xY8cOdO/e3TLmscces5qze/duuLq6Iikpyap9/vz5EELg448/tmoPDw9HSEiI5X3Pnj0xfvx4ZGdno66uDgDg5eVl6a+pqcHly5fRr18/tG/fHkePHm2Qe+bMmXB3d7fK6Obmht27dzuwF5omISEBJpMJ27Zts7Rt3boVtbW1mDp1arN9rlrwkDURkYQ333wTgYGBcHNzQ5cuXRAUFAQXl/+uZdzc3NCjRw+rOYWFhQgICEC7du2s2m+99VZL/x/179+/wecGBgaioqICly5dQteuXVFZWYm0tDRkZGSgpKTE6lx0WVlZg/nXb9PX1xfdunWze87ZGQYMGIC77roLmZmZmD59OoD/HK4eOnQo+vXr12yfqxYsyEREEkJDQy1XWdui1WqtCnRzmTNnDjIyMjB37lyEh4dDr9dDo9EgPj4eZrO52T+/qRISEvDkk0+iuLgYJpMJ+/fvx8qVK+WO1SqwIBMROVmvXr2Qk5OD8vJyq1Xy6dOnLf1/dObMmQbbyM/Ph7e3Nzp16gQA2LZtGxITE/H3v//dMqaqqgpXrlyxmeHMmTO4++67Le+vXr2KCxcuYOzYsQ7/d9WTuigsPj4e8+bNw7vvvovKykq4u7sjLi7upj+zLeA5ZCIiJxs7dizq6uoarAxXrFgBjUaDMWPGWLXn5eVZnQcuKipCVlYWRo8eDVdXVwCAq6trg1um3njjDcs55uutXbsWNTU1lverV69GbW1tg892hI+Pj92/CPj7+2PMmDF45513kJmZiejoaPj7+9/0Z7YFXCETETlZTEwM7r77bvztb39DQUEBbr/9duzZswdZWVmYO3cu+vbtazU+ODgYUVFRVrc9AUBqaqplzF//+lds2rQJer0et912G/Ly8pCTk2O5/ep61dXVuOeeezB58mT88MMPWLVqFYYNG4bY2Nib/u8LCQlBTk4Oli9fjoCAAPTp0wdhYWGW/oSEBEycOBEAsGjRopv+vLaCBZmIyMlcXFzw4Ycf4sUXX8TWrVuRkZGB3r17Y+nSpZg/f36D8SNHjkR4eDhSU1Px888/47bbbsP69esxePBgy5jXX38drq6uyMzMRFVVFf785z8jJycHUVFRNjOsXLkSmZmZePHFF1FTU4MpU6bgH//4h1MeFLF8+XLMnDnT8rWaiYmJVgU5JiYGHTp0gNlsdspfANoKjbj+GAgREdFNqK2tRUBAAGJiYrBu3Tq547QaPIdMREROtXPnTly6dAkJCQlyR2lVuEImIiKnOHDgAL777jssWrQI/v7+Nr+whOzjCpmIiJxi9erVeOyxx9C5c2ds3LhR7jitDlfIRERECsAVMhERkQI0+ban/3GZ1Jw5iNqUveb35Y5ARArD+5CJyCaz2Yzz58+jXbt2Trl3laitEkKgvLwcAQEBkt97zoJMRDadP38eBoNB7hhEqlFUVNTgyWB/xIJMRDbVPxShqKgIOp1O5jRErZfRaITBYGjwOM7rsSATkU31h6l1Oh0LMpETNHbqh1dZExERKQALMhERkQKwIBMRESkACzIREZECsCATEREpAAsyERGRArAgExERKQDvQyYiScEp2XDRejs0tyB9nJPTEKkXV8hEKnPkyBFER0dDp9OhXbt2GD16NL755hu5YxFRI7hCJlKRo0ePYtiwYTAYDEhJSYHZbMaqVaswcuRIHDx4EEFBQXJHJCI7WJCJVGTBggXw8vJCXl4e/Pz8AABTp05FYGAgnn/+eXzwwQcyJyQie3jImkhFvvzyS0RGRlqKMQB069YNI0eOxK5du3D16lUZ0xGRFBZkIhUxmUzw8vJq0O7t7Y3q6mocP35chlRE1BQ8ZE2kIkFBQdi/fz/q6urg6uoKAKiursaBAwcAACUlJXbnmkwmmEwmy3uj0di8YYnIClfIRCry+OOPIz8/H9OnT8fJkydx/PhxJCQk4MKFCwCAyspKu3PT0tKg1+stL4PB0FKxiQgsyESq8uijj+L555/H5s2bMXDgQAwaNAhnz57FM888AwDw9fW1Ozc5ORllZWWWV1FRUUvFJiKwIBOpzuLFi1FaWoovv/wS3333HQ4dOgSz2QwACAwMtDtPq9VCp9NZvYio5fAcMpEKdejQAcOGDbO8z8nJQY8ePTBgwAAZUxGRFK6QiVRu69atOHToEObOnQsXF/7KEykVV8hEKvLFF19g4cKFGD16NPz8/LB//35kZGQgOjoaTz75pNzxiEgCCzKRinTv3h2urq5YunQpysvL0adPH7z88suYN28e3Nz4606kZPwNJVKRvn37Ijs7W+4YROQAFmQiknQ8NYpXXBO1AF7hQUREpAAsyERERArAgkxERKQAPIdMRJKCU7LhovV2aG5B+jgnpyFSL66QiYiIFIAFmUiFzpw5g/j4ePTo0QPe3t4YMGAAFi5ciIqKCrmjEZEdPGRNpDJFRUUIDQ2FXq/H7Nmz0bFjR+Tl5SElJQVHjhxBVlaW3BGJyAYWZCKV2bRpE65cuYKvvvoKAwcOBADMnDkTZrMZGzduxO+//44OHTrInJKIrsdD1kQqYzQaAQBdunSxau/WrRtcXFzg4eEhRywiagRXyDLQhAyU7Dd72P/fUjLKR3LuiTmrJPtrRJ1kvxzuOT5Rst9n/AXJfnNVlTPjtHqjRo3CkiVLMH36dKSmpsLPzw/79u3D6tWrkZSUBB8f2z9DJpMJJpPJ8r6+sBNRy+AKmUhloqOjsWjRIuzduxdDhgxBz549ER8fjzlz5mDFihV256WlpUGv11teBoOhBVMTEQsykQr17t0bI0aMwNq1a/HBBx/g4YcfxiuvvIKVK1fanZOcnIyysjLLq6ioqAUTExEPWROpzJYtWzBz5kzk5+ejR48eAIAJEybAbDbj2WefxZQpU+Dn59dgnlarhVarbem4RPR/uEImUplVq1ZhyJAhlmJcLzY2FhUVFTh27JhMyYhICgsykcqUlpairq7hxXs1NTUAgNra2paORERNwIJMpDKBgYE4duwY8vPzrdrfffdduLi4YPDgwTIlIyIpPIfsIBF+u2T/mWn27/Vc8Zd3Jee6a+yvYCK9yiXn1gjpv2OZYZbsl8Pe4Pck++/Y9LBkf5/Hztvtq/v1skOZWrOnn34aH3/8MYYPH47Zs2fDz88Pu3btwscff4wZM2YgICBA7ohEZAMLMpHKjBgxAvv27cNLL72EVatW4fLly+jTpw8WL16MZ555Ru54RGQHCzKRCoWGhmL37t1O2dbx1CjodDqnbIuI7OM5ZCIiIgVgQSYiIlIAFmQiIiIF4DlkIpIUnJINF633Dc0pSB/XTGmI1IsrZCIiIgXgCtlB4uXfJPtPD9jeQknU75uItyX7o8Iet9un/Xfbuw952rRp2LBhg93+4uJidO/evQUTEVFTsCATqcysWbMQGRlp1SaEwKOPPorevXuzGBMpFAsykcqEh4cjPDzcqu2rr75CRUUFHnjgAZlSEVFjeA6ZqA3YvHkzNBoN7r//frmjEJEdLMhEKldTU4P33nsPERER6N27t9xxiMgOHrImUrns7Gxcvny50cPVJpMJJpPJ8t5oNDZ3NCL6A66QiVRu8+bNcHd3x+TJkyXHpaWlQa/XW14Gg6GFEhIRwIJMpGpXr15FVlYWoqKi4OfnJzk2OTkZZWVllldRUVELpSQigIesHVaS28jqYYDj286r0trte3j3I9KTNY1sXNx4nnpD78y325fRe4/jG6Zms3PnziZfXa3VaqHV2v/ZI6LmxRUykYplZmbC19cXsbGxckchokawIBOp1KVLl5CTk4P77rsP3t439l3URNTyWJCJVGrr1q2ora3ll4EQtRIsyEQqlZmZic6dOzf4Gk0iUiZe1EWkUnl5eU7ZzvHUKOh0Oqdsi4js4wqZiIhIAbhCdlDP9MOS/fe9N8XhbWuqa+z29T93wOHt3qwr/vbvY83Z305ybqRXucOf+5fv4yT7dZ+dsNtndvhTiYhaFlfIRERECsAVMhFJCk7JhovW8dumCtLHOTENkXpxhUxERKQALMhEKnX06FHExsaiY8eO8Pb2RnBwMP7xj3/IHYuI7OAhayIV2rNnD2JiYjBkyBAsWLAAvr6+OHv2LIqLi+WORkR2sCATqYzRaERCQgLGjRuHbdu2wcWFB8KIWgP+phKpzObNm1FaWorFixfDxcUF165dg9nMG8CIlI4rZAeJmmrJ/roffmyhJC2ndEKg3b5BHlmNzHb8sX7nz3eU7Pet+MnhbatRTk4OdDodSkpKcO+99yI/Px8+Pj548MEHsWLFCnh6esodkYhs4AqZSGXOnDmD2tpajB8/HlFRUfjggw/w8MMP46233sJDDz1kd57JZILRaLR6EVHL4QqZSGWuXr2KiooKPProo5arqidMmIDq6mqsWbMGCxcuRP/+/RvMS0tLQ2pqakvHJaL/wxUykcp4eXkBAKZMsf761vvvvx+A/YdOJCcno6yszPIqKipq3qBEZIUrZCKVCQgIwIkTJ9ClSxer9s6dOwMAfv/9d5vztFottFrHz/UT0c3hCplIZUJCQgAAJSUlVu3nz58HAHTq1KnFMxFR41iQiVRm8uTJAIB169ZZtf/rX/+Cm5sbRo0aJUMqImoMD1mTxaXHwiX7B0w9bbevi2vzHeq89Zlzkv11zfbJrdOQIUPw8MMP4+2330ZtbS1GjhyJ3NxcvP/++0hOTkZAQIDcEYnIBhZkIhV666230LNnT2RkZGDHjh3o1asXVqxYgblz58odjYjsYEEmUiF3d3ekpKQgJSVF7ihE1EQsyEQk6XhqFHQ6ndwxiFSPF3UREREpAAsyERGRAvCQNRFJCk7JhovWu8U/tyB9XIt/JpGcuEImIiJSAK6QVeaX2RGS/YmP7bbbN1W3THJuOxcPhzI1ZtGlOyX7hUn6UZdkLTc3F3fffbfNvry8PAwdOrSFExFRU7AgE6lUUlIS7rrrLqu2fv36yZSGiBrDgkykUsOHD8fEiRPljkFETcRzyEQqVl5ejtraWrljEFETsCATqdRDDz0EnU4HT09P3H333Th8+LDckYhIAg9ZE6mMh4cH/vd//xdjx46Fv78/Tp48iWXLlmH48OHYt28fhgwZYnOeyWSCyWSyvDcajS0VmYjAgkykOhEREYiI+O/V9rGxsZg4cSIGDx6M5ORkfPLJJzbnpaWlITU1taViEtF1WJAd5DowSLI//6EOdvtGDjvu7DgWuwxvSPabYZbodfy2ph9rpM9Txq2eb7ev545Sybnm8rMOZaL/6tevH8aPH4/t27ejrq4Orq6uDcYkJydj3rx5lvdGoxEGg6ElYxK1aSzIRG2EwWBAdXU1rl27ZvNhEVqtFlpt8z3Xmoik8aIuojbip59+gqenJ3x9feWOQkQ2sCATqcylS5catH377bf48MMPMXr0aLi48NeeSIl4yJpIZeLi4uDl5YWIiAh07twZJ0+exNq1a+Ht7Y309HS54xGRHSzIRCpz7733IjMzE8uXL4fRaESnTp0wYcIEpKSk8KsziRSMBZlIZZKSkpCUlCR3DCK6QSzIRCTpeGqUzauyici5WJAliD/fYbdvWsYOybnjfX51cpqmkueCnaQf4yT7uy/ZZ7evztlhiIhaIV5uSUREpAAsyERERArAQ9ZEJCk4JRsuWm+nbKsgfZxTtkOkRlwhExERKQALMpHKLV68GBqNBsHBwXJHISIJLMhEKlZcXIxXXnkFPj4+ckchokbwHDKRij311FMYOnQo6urq8Ouvct2KR0RNwYLsIFcIyX4XmQ4+uGsaPuf2j2qkYzvsk1ul78se/sATdvv0mfudHYcAfPHFF9i2bRuOHTuGOXPmyB2HiBrBgkykQnV1dZgzZw5mzJiBQYMGNWmOyWSCyWSyvDcajc0Vj4hsYEEmUqG33noLhYWFyMnJafKctLQ0pKamNmMqIpLCi7qIVOby5ct48cUXsWDBAnTq1KnJ85KTk1FWVmZ5FRUVNWNKIroeV8hEKvPCCy+gY8eON3zeWKvVQqvVNlMqImoMCzKRipw5cwZr167Fa6+9hvPnz1vaq6qqUFNTg4KCAuh0OnTs2FHGlERkCw9ZE6lISUkJzGYzkpKS0KdPH8vrwIEDyM/PR58+fbBw4UK5YxKRDVwhS9B8/Y3dvnX3RkvOfW6an92+ntnVknNdK2sl+5vLmenukv2no1e3UBJyVHBwMHbsaHgL2gsvvIDy8nK8/vrr6Nu3rwzJiKgxLMhEKuLv74977723Qftrr70GADb7iEgZeMiaiIhIAbhCJmoDcnNzHZ57PDUKOp3OeWGIyCaukImIiBSABZmIiEgBWJCJiIgUgOeQiUhScEo2XLTezbLtgvRxzbJdotaIBdlBdSfzJftveaaFgjjRrWca+d5j6VuviYjoJvCQNZHKnDhxApMmTcItt9wCb29v+Pv7Y8SIEfjoo4/kjkZEErhCJlKZwsJClJeXIzExEQEBAaioqMAHH3yA2NhYrFmzBjNnzpQ7IhHZwIJMpDJjx47F2LFjrdpmz56NkJAQLF++nAWZSKF4yJqoDXB1dYXBYMCVK1fkjkJEdnCFTKRS165dQ2VlJcrKyvDhhx/i448/RlxcnNyxiMgOFmQilZo/fz7WrFkDAHBxccGECROwcuVKu+NNJhNMJpPlvdFobPaMRPRfLMhkUTqhn9wRyInmzp2LiRMn4vz583jvvfdQV1eH6mr7j/5MS0tDampqCyYkoj/iOWQilRowYAAiIyORkJCAXbt24erVq4iJiYEQwub45ORklJWVWV5FRUUtnJiobWNBJmojJk6ciEOHDiE/3/aX2mi1Wuh0OqsXEbUcFmSiNqKyshIAUFZWJnMSIrKFBZlIZX755ZcGbTU1Ndi4cSO8vLxw2223yZCKiBrDi7qIVGbWrFkwGo0YMWIEunfvjosXLyIzMxOnT5/G3//+d/j6+sodkYhsYEEmUpm4uDisW7cOq1evxuXLl9GuXTuEhIRgyZIliI2NlTseEdnBgkykMvHx8YiPj3fa9o6nRvECL6IWoOqCrNFqJfuvTBoi2d8h64TdPnN5uUOZ5HZhfoTdvqykVxuZLb0/iYjIcbyoi4iISAFYkImIiBRA1YesiejmBadkw0Xr3eTxBenjmjENkXpxhUxERKQALMhEKnPo0CHMnj0bAwcOhI+PD3r27InJkyfb/cpMIlIGHrImUpklS5bg66+/xqRJkzB48GBcvHgRK1euxJ133on9+/cjODhY7ohEZEOrL8hVMaF2+/RP/Sw59/N+b0j233doiv3OH+S57cmtW1fJ/pKJt0j2b52zzG5fgJvjtzWV1pkk+90rbT9hiJxv3rx52Lx5Mzw8PCxtcXFxGDRoENLT0/HOO+/ImI6I7Gn1BZmIrEVENLzXvH///hg4cCBOnTolQyIiagqeQyZqA4QQKC0thb+/v9xRiMgOFmSiNiAzMxMlJSWIi4uzO8ZkMsFoNFq9iKjlsCATqdzp06fxxBNPIDw8HImJiXbHpaWlQa/XW14Gg6EFUxIRCzKRil28eBHjxo2DXq/Htm3b4OrqandscnIyysrKLK+ioqIWTEpEvKiLSKXKysowZswYXLlyBV9++SUCAgIkx2u1WmgbeSALETUfFmQiFaqqqkJMTAzy8/ORk5OD2267Te5IRNSIVl+QoxZ/brdvvt/xm9r26eclngF7Neymtu2o+Ig8yf6dnf8t2W+Gu8OfnVgQZbfvx4wgybl+26Vzk/PU1dUhLi4OeXl5yMrKQnh4uNyRiKgJWn1BJiJr8+fPx4cffoiYmBj89ttvDb4IZOrUqTIlIyIpLMhEKvPNN98AAD766CN89NFHDfpZkImUiQWZSGVyc3Odur3jqVHQ6SRO3xCRU/C2JyIiIgVgQSYiIlIAFmQiIiIF4DlkIpIUnJINF62307ZXkD7OadsiUhMWZAmnItfIHcEB0gc98qrsfxPTIwcSJOf2e+SM3T6/a7zPmIjoZvCQNZHKXL16FSkpKYiOjkbHjh2h0Wiwfv16uWMRUSNYkIlU5tdff8XChQtx6tQp3H777XLHIaIm4iFrIpXp1q0bLly4gK5du+Lw4cO466675I5ERE3AFTKRymi1WnTt2lXuGER0g1iQiYiIFICHrIkIAGAymWAymSzvjUajjGmI2p5WX5D/X9Kf7fZtfDxUcu63f37b2XGc4h2jwW7fhZr2knPfPmp/fwBAv3/W2e275etvJOeaJXuptUtLS0NqaqrcMYjaLB6yJiIAQHJyMsrKyiyvoqIiuSMRtSmtfoVMRM6h1Wqh1dr/4hgial5cIRMRESkACzIREZEC8JA1kQqtXLkSV65cwfnz5wEAH330EYqLiwEAc+bMgV6vlzMeEdnAgkykQsuWLUNhYaHl/fbt27F9+3YAwNSpU1mQiRSIBZlIhQoKCuSOQEQ3qNUXZNfco3b7+hyUfoZrSNKTkv0bZr1mty/YQyM59y/fx9ntK8uV/lrDXltL7PbVniu02wcA/XFEsp/oRh1PjYJOp5M7BpHq8aIuIiIiBWBBJiIiUoBWf8iaiJpXcEo2XLTSp3/sKUgf5+Q0ROrFFTIREZECsCATqZDJZMKzzz6LgIAAeHl5ISwsDHv37pU7FhFJYEEmUqFp06Zh+fLleOCBB/D666/D1dUVY8eOxVdffSV3NCKyQyOEEE0Z+D8uk5o7C1Gbsdf8frNt++DBgwgLC8PSpUvx1FNPAQCqqqoQHByMzp07Y9++fU3ajtFohF6vh2HuezyHTHQT6n+XysrKJG8h5AqZSGW2bdsGV1dXzJw509Lm6emJ6dOnIy8vj49VJFIoFmQilTl27BgCAwMb/E08NDQUAPDNN9/IkIqIGsPbnohU5sKFC+jWrVuD9vq2+gdOXM9kMsFkMlneG43G5glIRDZxhUykMpWVldBqtQ3aPT09Lf22pKWlQa/XW14Gg6FZcxKRNRZkIpXx8vKyWunWq6qqsvTbkpycjLKyMsuL55qJWhYPWROpTLdu3VBS0vABJRcuXAAABAQE2Jyn1WptrqyJqGVwhUykMnfccQfy8/MbnAM+cOCApZ+IlIcFmUhlJk6ciLq6Oqxdu9bSZjKZkJGRgbCwMJ4bJlIoHrImUpmwsDBMmjQJycnJ+OWXX9CvXz9s2LABBQUFWLdundzxiMgOFmQiFdq4cSMWLFiATZs24ffff8fgwYOxa9cujBgxQu5oRGQHCzKRCnl6emLp0qVYunSp3FGIqIlYkIlI0vHUKMnv3yUi5+BFXURERArAgkxERKQALMhEREQKwIJMRESkACzIRERECsCCTEREpAAsyERERArA+5CJyCYhBAA0eEgFEd2Y+t+h+t8pe1iQicimy5cvAwAfRkHkJOXl5dDr9Xb7WZCJyKaOHTsCAH7++WfJP0SUymg0wmAwoKioqFV+0xjzy8uZ+YUQKC8vt/ss8npNLsh7ze/fVCAial1cXP5ziYler2+Vf6DW0+l0zC8j5v+Ppvyllhd1ERERKQALMhERkQKwIBORTVqtFikpKdBqtXJHcQjzy4v5b5xGNHYdNhERETU7rpCJiIgUgAWZiIhIAViQiYiIFIAFmYiISAFYkInaiDfffBO9e/eGp6cnwsLCcPDgQcnx77//PgYMGABPT08MGjQIu3fvtuoXQuDFF19Et27d4OXlhcjISJw5c0YR+f/5z39i+PDh6NChAzp06IDIyMgG46dNmwaNRmP1io6OVkT+9evXN8jm6elpNUbJ+3/UqFEN8ms0GowbN84ypiX3/xdffIGYmBgEBARAo9Fg586djc7Jzc3FnXfeCa1Wi379+mH9+vUNxtzo71SjBBGp3pYtW4SHh4d4++23xYkTJ8Qjjzwi2rdvL0pLS22O//rrr4Wrq6t49dVXxcmTJ8ULL7wg3N3dxffff28Zk56eLvR6vdi5c6f49ttvRWxsrOjTp4+orKyUPf/9998v3nzzTXHs2DFx6tQpMW3aNKHX60VxcbFlTGJiooiOjhYXLlywvH777TenZ3ckf0ZGhtDpdFbZLl68aDVGyfv/8uXLVtmPHz8uXF1dRUZGhmVMS+7/3bt3i7/97W9i+/btAoDYsWOH5PiffvpJeHt7i3nz5omTJ0+KN954Q7i6uopPPvnEMuZG90lTsCATtQGhoaHiiSeesLyvq6sTAQEBIi0tzeb4yZMni3Hjxlm1hYWFiVmzZgkhhDCbzaJr165i6dKllv4rV64IrVYr3n33XdnzX6+2tla0a9dObNiwwdKWmJgoxo8f7+yoNt1o/oyMDKHX6+1ur7Xt/xUrVoh27dqJq1evWtpacv//UVMK8jPPPCMGDhxo1RYXFyeioqIs7292n9jCQ9ZEKlddXY0jR44gMjLS0ubi4oLIyEjk5eXZnJOXl2c1HgCioqIs48+dO4eLFy9ajdHr9QgLC7O7zZbMf72KigrU1NRYHphRLzc3F507d0ZQUBAee+wxyxOunMnR/FevXkWvXr1gMBgwfvx4nDhxwtLX2vb/unXrEB8fDx8fH6v2ltj/jmjs598Z+8QWFmQilfv1119RV1eHLl26WLV36dIFFy9etDnn4sWLkuPr/3kj23SUI/mv9+yzzyIgIMDqD9Do6Ghs3LgRn376KZYsWYLPP/8cY8aMQV1dnez5g4KC8PbbbyMrKwvvvPMOzGYzIiIiUFxcDKB17f+DBw/i+PHjmDFjhlV7S+1/R9j7+TcajaisrHTKz6QtfPwiEalaeno6tmzZgtzcXKsLo+Lj4y3/PmjQIAwePBh9+/ZFbm4u7rnnHjmiWoSHhyM8PNzyPiIiArfeeivWrFmDRYsWyZjsxq1btw6DBg1CaGioVbuS979cuEImUjl/f3+4urqitLTUqr20tBRdu3a1Oadr166S4+v/eSPbdJQj+estW7YM6enp2LNnDwYPHiw59pZbboG/vz9+/PHHm878RzeTv567uzuGDBliydZa9v+1a9ewZcsWTJ8+vdHPaa797wh7P/86nQ5eXl5O+X9qCwsykcp5eHggJCQEn376qaXNbDbj008/tVqF/VF4eLjVeADYu3evZXyfPn3QtWtXqzFGoxEHDhywu82WzA8Ar776KhYtWoRPPvkEf/rTnxr9nOLiYly+fBndunVzSu56jub/o7q6Onz//feWbK1h/wP/uXXOZDJh6tSpjX5Oc+1/RzT28++M/6c2OXw5GBG1Glu2bBFarVasX79enDx5UsycOVO0b9/ecivNgw8+KJ577jnL+K+//lq4ubmJZcuWiVOnTomUlBSbtz21b99eZGVlie+++06MHz++WW+7uZH86enpwsPDQ2zbts3qtpry8nIhhBDl5eXiqaeeEnl5eeLcuXMiJydH3HnnnaJ///6iqqpK9vypqakiOztbnD17Vhw5ckTEx8cLT09PceLECav/RqXu/3rDhg0TcXFxDdpbev+Xl5eLY8eOiWPHjgkAYvny5eLYsWOisLBQCCHEc889Jx588EHL+Prbnp5++mlx6tQp8eabb9q87UlqnziCBZmojXjjjTdEz549hYeHhwgNDRX79++39I0cOVIkJiZajX/vvfdEYGCg8PDwEAMHDhT//ve/rfrNZrNYsGCB6NKli9BqteKee+4RP/zwgyLy9+rVSwBo8EpJSRFCCFFRUSFGjx4tOnXqJNzd3UWvXr3EI488clN/mDoz/9y5cy1ju3TpIsaOHSuOHj1qtT0l738hhDh9+rQAIPbs2dNgWy29/z/77DObPw/1mRMTE8XIkSMbzLnjjjuEh4eHuOWWW6zuoa4ntU8cwccvEhERKQDPIRMRESkACzIREZECsCATEREpAAsyERGRArAgExERKQALMhERkQKwIBMRESkACzIREZECsCATEREpAAsyERGRArAgExERKQALMhERkQL8f+vepT7qnF66AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 600x700 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "img = x_train[7:8,:]\n",
    "logits = model(img)\n",
    "\n",
    "# Predict the class from the network output\n",
    "prediction = tf.nn.softmax(logits).numpy()\n",
    "\n",
    "view_classification(img[0], prediction[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Back Propagation\n",
    "We need to now train the network to adjust its weights by first calculating Cross Entropy Loss and then back propagating the error to adjust weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create loss_fn\n",
    "loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train network\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss=loss_fn,\n",
    "              metrics=['accuracy'])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/2\n",
      "1875/1875 [==============================] - 13s 7ms/step - loss: 0.2118 - accuracy: 0.9370\n",
      "Epoch 2/2\n",
      "1875/1875 [==============================] - 11s 6ms/step - loss: 0.0905 - accuracy: 0.9728\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.src.callbacks.History at 0x7f24307bca60>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train model\n",
    "model.fit(x_train, y_train, epochs=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 - 1s - loss: 0.0903 - accuracy: 0.9711 - 895ms/epoch - 3ms/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.09025799483060837, 0.9710999727249146]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# evaluate model\n",
    "model.evaluate(x_test,  y_test, verbose=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAERCAYAAACq8dRTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAl50lEQVR4nO3deVxU9f4/8NcwwLDJqOACOi6pYImWUSCkYsUVxAuWVwXLwNKwRclHttFVCc3QMm0xTe81XMK0TKW6lkq/aBN3W9zCTAlQySgZEBiW+fz+uF/mNjJzQATO4fB6Ph7zqPl8PufMyyPjm89ZNUIIASIiIpKVg9wBiIiIiAWZiIhIEViQiYiIFIAFmYiISAFYkImIiBSABZmIiEgBWJCJiIgUgAWZiIhIAViQiYiIFIAFmYhIRhqNBjNnzmy29a1btw4ajQaHDh1qcOyoUaMwatQoy/tz585Bo9Fg3bp1lrYXXngBGo2m2fKRfSzIREQ21BW2upeLiwv8/Pwwc+ZMFBUVyR1PVi+99BJ27NghdwzVYUEmIpKwYMECbNy4EStWrEBoaChWrVqFkJAQlJeXyx3tuu3evRu7d++WHDN37lxUVFRYtbEgtwxHuQMQESnZmDFjcNtttwEApk+fDi8vLyxbtgyZmZmYPHlyvfFXrlyBu7t7a8dsEmdn5wbHODo6wtGRpaI1cIZMRHQN7rrrLgDA2bNnMXXqVHh4eODMmTOIiopChw4dcP/99wP4b2GeM2cODAYDdDod/P39sXTpUth7wF5GRgb8/f3h4uKCwMBAfPXVV1b9eXl5eOyxx+Dv7w9XV1d4eXlh4sSJOHfunM31lZeXY8aMGfDy8oKnpyfi4+Px559/Wo25+hiyLVcfQ9ZoNLhy5QrWr19v2Z0/depUfPHFF9BoNNi+fXu9dWzatAkajQY5OTmSn9Xe8dceIqJrcObMGQCAl5cXAKCmpgYREREYPnw4li5dCjc3NwghEBMTgy+++ALTpk3DLbfcgl27duHpp59GYWEhli9fbrXOL7/8Elu2bEFSUhJ0Oh1WrlyJyMhIHDhwAAEBAQCAgwcPYu/evYiLi0PPnj1x7tw5rFq1CqNGjcKJEyfg5uZmtc6ZM2eiY8eOeOGFF/DTTz9h1apVyMvLQ3Z29nWdpLVx40ZMnz4dQUFBSExMBAD069cPw4YNg8FgQEZGBu69916rZTIyMtCvXz+EhIQ0+XPbBUFERPWkp6cLACIrK0tcunRJ5Ofni82bNwsvLy/h6uoqCgoKREJCggAgnnvuOatld+zYIQCIF1980ap9woQJQqPRiJ9//tnSBkAAEIcOHbK05eXlCRcXF3Hvvfda2srLy+tlzMnJEQDEhg0b6uUODAwUVVVVlvaXX35ZABCZmZmWtrCwMBEWFmZ5f/bsWQFApKenW9pSUlLE1aXC3d1dJCQk1MuTnJwsdDqduHz5sqXtt99+E46OjiIlJaXeeLLGXdZERBLCw8PRpUsXGAwGxMXFwcPDA9u3b0ePHj0sYx599FGrZXbu3AmtVoukpCSr9jlz5kAIgU8//dSqPSQkBIGBgZb3vXr1wrhx47Br1y7U1tYCAFxdXS391dXVKC4uRv/+/dGxY0ccOXKkXu7ExEQ4OTlZZXR0dMTOnTubsBUaJz4+HiaTCVu3brW0bdmyBTU1NZgyZUqLfa5acJc1EZGEt956C35+fnB0dES3bt3g7+8PB4f/zWUcHR3Rs2dPq2Xy8vLg6+uLDh06WLXfeOONlv6/GjBgQL3P9fPzQ3l5OS5duoTu3bujoqICaWlpSE9PR2FhodWx6JKSknrLX71ODw8P+Pj42D3m3BwGDhyI22+/HRkZGZg2bRqA/+6uHjZsGPr3799in6sWLMhERBKCgoIsZ1nbotPprAp0S5k1axbS09Mxe/ZshISEQK/XQ6PRIC4uDmazucU/v7Hi4+PxxBNPoKCgACaTCfv27cOKFSvkjtUmsCATETWz3r17IysrC6WlpVaz5FOnTln6/+r06dP11pGbmws3Nzd06dIFALB161YkJCTg1VdftYyprKzE5cuXbWY4ffo07rzzTsv7srIyXLhwAVFRUU3+c9WROiksLi4OTz75JN577z1UVFTAyckJsbGx1/2Z7QGPIRMRNbOoqCjU1tbWmxkuX74cGo0GY8aMsWrPycmxOg6cn5+PzMxMjB49GlqtFgCg1WrrXTL15ptvWo4xX23NmjWorq62vF+1ahVqamrqfXZTuLu72/1FwNvbG2PGjMG7776LjIwMREZGwtvb+7o/sz3gDJmIqJlFR0fjzjvvxD//+U+cO3cON998M3bv3o3MzEzMnj0b/fr1sxofEBCAiIgIq8ueACA1NdUy5u9//zs2btwIvV6Pm266CTk5OcjKyrJcfnW1qqoq3H333Zg0aRJ++uknrFy5EsOHD0dMTMx1//kCAwORlZWFZcuWwdfXF3379kVwcLClPz4+HhMmTAAALFy48Lo/r71gQSYiamYODg746KOPMH/+fGzZsgXp6eno06cPXnnlFcyZM6fe+LCwMISEhCA1NRW//vorbrrpJqxbtw5DhgyxjHn99deh1WqRkZGByspK3HHHHcjKykJERITNDCtWrEBGRgbmz5+P6upqTJ48GW+88UazPChi2bJlSExMtNxWMyEhwaogR0dHo1OnTjCbzc3yC0B7oRFX7wMhIiK6DjU1NfD19UV0dDTWrl0rd5w2g8eQiYioWe3YsQOXLl1CfHy83FHaFM6QiYioWezfvx8//PADFi5cCG9vb5s3LCH7OEMmIqJmsWrVKjz66KPo2rUrNmzYIHecNoczZCIiIgXgDJmIiEgBGn3Z098cJrZkDqJ2ZY/5A7kjEJHC8DpkIrLJbDbj/Pnz6NChQ7Ncu0rUXgkhUFpaCl9fX8n7nrMgE5FN58+fh8FgkDsGkWrk5+fXezLYX7EgE5FNdQ9FyM/Ph6enp8xpiNouo9EIg8FQ73GcV2NBJiKb6nZTe3p6siATNYOGDv3wLGsiIiIFYEEmIiJSABZkIiIiBWBBJiIiUgAWZCIiIgVgQSYiIlIAFmQiIiIFYEEmIiJSABZkIpU5fPgwIiMj4enpiQ4dOmD06NH47rvv5I5FRA3gnbqIVOTIkSMYPnw4DAYDUlJSYDabsXLlSoSFheHAgQPw9/eXOyIR2cGCTKQi8+bNg6urK3JycuDl5QUAmDJlCvz8/PD888/jww8/lDkhEdnDXdZEKvL1118jPDzcUowBwMfHB2FhYfjkk09QVlYmYzoiksKCTKQiJpMJrq6u9drd3NxQVVWFY8eOyZCKiBqDu6yJVMTf3x/79u1DbW0ttFotAKCqqgr79+8HABQWFtpd1mQywWQyWd4bjcaWDUtEVjhDJlKRxx57DLm5uZg2bRpOnDiBY8eOIT4+HhcuXAAAVFRU2F02LS0Ner3e8jIYDK0Vm4jAgkykKo888gief/55bNq0CYMGDcLgwYNx5swZPPPMMwAADw8Pu8smJyejpKTE8srPz2+t2EQEFmQi1Vm0aBGKiorw9ddf44cffsDBgwdhNpsBAH5+fnaX0+l08PT0tHoRUevhMWQiFerUqROGDx9ueZ+VlYWePXti4MCBMqYiIimcIROp3JYtW3Dw4EHMnj0bDg78yhMpFWfIRCry1VdfYcGCBRg9ejS8vLywb98+pKenIzIyEk888YTc8YhIAgsykYr06NEDWq0Wr7zyCkpLS9G3b1+8+OKLePLJJ+HoyK87kZLxG0qkIv369cOuXbvkjkFETcADSkRERArAgkxERKQALMhEREQKwIJMRESkACzIRERECsCCTEREpAAsyEQqdPr0acTFxaFnz55wc3PDwIEDsWDBApSXl8sdjYjs4HXIRCqTn5+PoKAg6PV6zJw5E507d0ZOTg5SUlJw+PBhZGZmyh2RiGxgQSZSmY0bN+Ly5cv45ptvMGjQIABAYmIizGYzNmzYgD///BOdOnWSOSURXY27rIlUxmg0AgC6detm1e7j4wMHBwc4OzvLEYuIGsAZsgw0gYMk+83O9v9aCke5Sy57fNZKyf5qUSvZL4e7j02Q7Hcfd0Gy31xZ2Zxx2rxRo0ZhyZIlmDZtGlJTU+Hl5YW9e/di1apVSEpKgru77Z8hk8kEk8lkeV9X2ImodXCGTKQykZGRWLhwIfbs2YOhQ4eiV69eiIuLw6xZs7B8+XK7y6WlpUGv11teBoOhFVMTEQsykQr16dMHI0eOxJo1a/Dhhx/ioYcewksvvYQVK1bYXSY5ORklJSWWV35+fismJiLusiZSmc2bNyMxMRG5ubno2bMnAGD8+PEwm8149tlnMXnyZHh5edVbTqfTQafTtXZcIvo/nCETqczKlSsxdOhQSzGuExMTg/Lychw9elSmZEQkhQWZSGWKiopQW1v/5L3q6moAQE1NTWtHIqJGYEEmUhk/Pz8cPXoUubm5Vu3vvfceHBwcMGTIEJmSEZEUHkNuIhFys2T/6an2r/Vcftd7kss6aezPYMJdSyWXrRbSv2OZYZbsl8OegPcl+2/Z+JBkf99Hz9vtq/29uEmZ2rKnn34an376KUaMGIGZM2fCy8sLn3zyCT799FNMnz4dvr6+ckckIhtYkIlUZuTIkdi7dy9eeOEFrFy5EsXFxejbty8WLVqEZ555Ru54RGQHCzKRCgUFBWHnzp1yxyCia8BjyERERArAgkxERKQALMhEREQKwIJMRESkACzIRERECsCzrJtIvPiHZP+pgdtaKYn6fRf6jmR/RPBjdvt0/2l/1yFPnToV69evt9tfUFCAHj16tGIiImoMFmQilZkxYwbCw8Ot2oQQeOSRR9CnTx8WYyKFYkEmUpmQkBCEhIRYtX3zzTcoLy/H/fffL1MqImoIjyETtQObNm2CRqPBfffdJ3cUIrKDBZlI5aqrq/H+++8jNDQUffr0kTsOEdnBXdZEKrdr1y4UFxc3uLvaZDLBZDJZ3huNxpaORkR/wRkykcpt2rQJTk5OmDRpkuS4tLQ06PV6y8tgMLRSQiICWJCJVK2srAyZmZmIiIiAl5eX5Njk5GSUlJRYXvn5+a2UkogA7rJussLsBmYPA5u+7pxKnd2+h3Y+LL2wpoGVi2vPU2fYrbl2+9L77G76iqnF7Nixo9FnV+t0Ouh09n/2iKhlcYZMpGIZGRnw8PBATEyM3FGIqAEsyEQqdenSJWRlZeHee++Fm5ub3HGIqAEsyEQqtWXLFtTU1PBmIERtBAsykUplZGSga9eu9W6jSUTKxJO6iFQqJydH7ghEdA04QyYiIlIAzpCbqNfiQ5L9974/ucnr1lRV2+0bcHZ/k9d7vS5727+ONWtfB8llw11Lm/y5d/0YK9nv+cVxu33mJn8qEVHr4gyZiIhIAViQiYiIFIAFmYiISAFYkIlU6siRI4iJiUHnzp3h5uaGgIAAvPHGG3LHIiI7eFIXkQrt3r0b0dHRGDp0KObNmwcPDw+cOXMGBQUFckcjIjtYkIlUxmg0Ij4+HmPHjsXWrVvh4MAdYURtAb+pRCqzadMmFBUVYdGiRXBwcMCVK1dgNvMCMCKl4wy5iUR1lWR/7U8/t1KS1lM03s9u32DnzAaWbvpj/c6f7yzZ71H+S5PXrUZZWVnw9PREYWEh7rnnHuTm5sLd3R0PPPAAli9fDhcXF7kjEpENnCETqczp06dRU1ODcePGISIiAh9++CEeeughvP3223jwwQftLmcymWA0Gq1eRNR6OEMmUpmysjKUl5fjkUcesZxVPX78eFRVVWH16tVYsGABBgwYUG+5tLQ0pKamtnZcIvo/nCETqYyrqysAYPJk69u33nfffQDsP3QiOTkZJSUllld+fn7LBiUiK5whE6mMr68vjh8/jm7dulm1d+3aFQDw559/2lxOp9NBp2v6sX4iuj6cIROpTGBgIACgsLDQqv38+fMAgC5durR6JiJqGAsykcpMmjQJALB27Vqr9n//+99wdHTEqFGjZEhFRA3hLmuyuPRoiGT/wCmn7PZ107bcrs4bnzkr2V/bYp/cNg0dOhQPPfQQ3nnnHdTU1CAsLAzZ2dn44IMPkJycDF9fX7kjEpENLMhEKvT222+jV69eSE9Px/bt29G7d28sX74cs2fPljsaEdnBgkykQk5OTkhJSUFKSorcUYiokXgMmYiISAFYkImIiBSABZmIiEgBWJCJiIgUgAWZiIhIAXiWtcr8NjNUsj/h0Z12+6Z4LpVctoODc5MyNWThpVsl+4VJ+lGXZC07Oxt33nmnzb6cnBwMGzaslRMRUWOwIBOpVFJSEm6//Xartv79+8uUhogawoJMpFIjRozAhAkT5I5BRI3EY8hEKlZaWoqamhq5YxBRI7AgE6nUgw8+CE9PT7i4uODOO+/EoUOH5I5ERBK4y5pIZZydnfGPf/wDUVFR8Pb2xokTJ7B06VKMGDECe/fuxdChQ20uZzKZYDKZLO+NRmNrRSYisCATqU5oaChCQ/93tn1MTAwmTJiAIUOGIDk5GZ999pnN5dLS0pCamtpaMYnoKhohhGjMwL85TGzpLG2KdpC/ZH/ug53s9oUNP9bccSzeNnwp2W+GuUU+9+dq6eOUsavm2O3rtb1Ictna3DNNyqRke8wftPpnTp48Gdu2bUN5eTm0Wm29flszZIPBgJKSEnh6erZmVCJVMRqN0Ov1DX6XOEMmaicMBgOqqqpw5coVm/8o6HQ66HQt91xrIpLGk7qI2olffvkFLi4u8PDwkDsKEdnAgkykMpcuXarX9v333+Ojjz7C6NGj4eDArz2REnGXNZHKxMbGwtXVFaGhoejatStOnDiBNWvWwM3NDYsXL5Y7HhHZwYJMpDL33HMPMjIysGzZMhiNRnTp0gXjx49HSkoKb51JpGAsyEQqk5SUhKSkJLljENE14sEkIiIiBeAMWYK44xa7fVPTt0suO87992ZO01jy/I6V9HOsZH+PJXvt9tU2dxgiojaIM2QiIiIFYEEmIiJSABZkIiIiBWBBJiIiUgAWZCKVW7RoETQaDQICAuSOQkQSWJCJVKygoAAvvfQS3N3d5Y5CRA3gZU9EKvbUU09h2LBhqK2txe+/y3UpHhE1BgtyE2kh/RhpB5l2Pjhp6j/n9q+qG/X062v32Y3S12WPuP9xu336jH3NHYcAfPXVV9i6dSuOHj2KWbNmyR2HiBrAgkykQrW1tZg1axamT5+OwYMHN2oZk8kEk8lkeW80GlsqHhHZwIJMpEJvv/028vLykJWV1ehl0tLSkJqa2oKpiEgKT+oiUpni4mLMnz8f8+bNQ5cuXRq9XHJyMkpKSiyv/Pz8FkxJRFfjDJlIZebOnYvOnTtf83FjnU4HnU7XQqmIqCEsyEQqcvr0aaxZswavvfYazp8/b2mvrKxEdXU1zp07B09PT3Tu3FnGlERkC3dZE6lIYWEhzGYzkpKS0LdvX8tr//79yM3NRd++fbFgwQK5YxKRDZwhS9B8+53dvrX3REou+9xUL7t9vXZVSS6rraiR7G8pp6c5SfafilzVSkmoqQICArB9e/1L0ObOnYvS0lK8/vrr6NevnwzJiKghLMhEKuLt7Y177rmnXvtrr70GADb7iEgZuMuaiIhIAThDJmoHsrOz5Y5ARA3gDJmIiEgBWJCJiIgUgAWZiIhIAXgMmYgkBaTsgoPOTe4YRK3u3OKxrfp5LMhNVHsiV7L/hmdaKUgzuvF0A/c9lr70moiIrgN3WROpzPHjxzFx4kTccMMNcHNzg7e3N0aOHImPP/5Y7mhEJIEzZCKVycvLQ2lpKRISEuDr64vy8nJ8+OGHiImJwerVq5GYmCh3RCKygQWZSGWioqIQFRVl1TZz5kwEBgZi2bJlLMhECsVd1kTtgFarhcFgwOXLl+WOQkR2cIZMpFJXrlxBRUUFSkpK8NFHH+HTTz9FbGys3LGIyA4WZCKVmjNnDlavXg0AcHBwwPjx47FixQq7400mE0wmk+W90Whs8YxE9D8syGRRNL6/3BGoGc2ePRsTJkzA+fPn8f7776O2thZVVfYf/ZmWlobU1NRWTEhEf8VjyEQqNXDgQISHhyM+Ph6ffPIJysrKEB0dDSGEzfHJyckoKSmxvPLz81s5MVH7xoJM1E5MmDABBw8eRG6u7Zva6HQ6eHp6Wr2IqPWwIBO1ExUVFQCAkpISmZMQkS0syEQq89tvv9Vrq66uxoYNG+Dq6oqbbrpJhlRE1BCe1EWkMjNmzIDRaMTIkSPRo0cPXLx4ERkZGTh16hReffVVeHh4yB2RiGxgQSZSmdjYWKxduxarVq1CcXExOnTogMDAQCxZsgQxMTFyxyMiO1iQiVQmLi4OcXFxzba+Y6kRPMGLqBWouiBrdDrJ/ssTh0r2d8o8brfPXFrapExyuzAn1G5fZtLLDSwtvT2JiKjpeFIXERGRArAgExERKQALMhERkQKwIBMRESkACzKRyhw8eBAzZ87EoEGD4O7ujl69emHSpEl2b5lJRMqg6rOsidqjJUuW4Ntvv8XEiRMxZMgQXLx4EStWrMCtt96Kffv2ISAgQO6IRGRDmy/IldFBdvv0T/0queyX/d+U7L/34GT7nT/Jc9mTo093yf7CCTdI9m+ZtdRun69j0y9rKqo1SfY7Vdh+whA1vyeffBKbNm2Cs7OzpS02NhaDBw/G4sWL8e6778qYjojsafMFmYishYbWv9Z8wIABGDRoEE6ePClDIiJqDB5DJmoHhBAoKiqCt7e33FGIyA4WZKJ2ICMjA4WFhYiNjbU7xmQywWg0Wr2IqPWwIBOp3KlTp/D4448jJCQECQkJdselpaVBr9dbXgaDoRVTEhELMpGKXbx4EWPHjoVer8fWrVuh1Wrtjk1OTkZJSYnllZ+f34pJiYgndRGpVElJCcaMGYPLly/j66+/hq+vr+R4nU4HXQMPZCGilsOCTKRClZWViI6ORm5uLrKysnDTTTfJHYmIGtDmC3LEoi/t9s3xOnZd6z71vMQzYMuCr2vdTRUXmiPZv6PrfyT7zXBq8mcnnIuw2/dzur/ksl7bpHNT86mtrUVsbCxycnKQmZmJkJAQuSMRUSO0+YJMRNbmzJmDjz76CNHR0fjjjz/q3QhkypQpMiUjIiksyEQq89133wEAPv74Y3z88cf1+lmQiZSJBZlIZbKzs+WOQERNwMueiIiIFIAFmYiISAFYkImIiBSABZmIiEgBeFKXhJPhq+WO0ATSv2PlVNq/E9PD++Mll+3/8Gm7fV5XeJ0xEdH14AyZSGXKysqQkpKCyMhIdO7cGRqNBuvWrZM7FhE1gAWZSGV+//13LFiwACdPnsTNN98sdxwiaiTusiZSGR8fH1y4cAHdu3fHoUOHcPvtt8sdiYgagTNkIpXR6XTo3r273DGI6BqxIBMRESkAd1kTEQDAZDLBZDJZ3huNRhnTELU/bb4g/7+kO+z2bXgsSHLZ7+94p7njNIt3jQa7fReqO0ou+84R+9sDAPr/q9Zu3w3ffie5rFmyl9q6tLQ0pKamyh2DqN3iLmsiAgAkJyejpKTE8srPz5c7ElG70uZnyETUPHQ6HXQ6+zeOIaKWxRkyERGRArAgExERKQB3WROp0IoVK3D58mWcP38eAPDxxx+joKAAADBr1izo9Xo54xGRDSzIRCq0dOlS5OXlWd5v27YN27ZtAwBMmTKFBZlIgViQiVTo3LlzckcgomvU5guyNvuI3b6+B9wklw1MekKyf/2M1+z2BThrJJe968dYu30l2dK3Ney9pdBuX83ZPLt9ADAAhyX7iYhImXhSFxERkQKwIBMRESkACzIREZECsCATEREpAAsykQqZTCY8++yz8PX1haurK4KDg7Fnzx65YxGRBBZkIhWaOnUqli1bhvvvvx+vv/46tFotoqKi8M0338gdjYjs0AghRGMG/s1hYktnIWo39pg/aLF1HzhwAMHBwXjllVfw1FNPAQAqKysREBCArl27Yu/evY1aj9FohF6vR0lJCTw9PVssL5HaNfa7xBkykcps3boVWq0WiYmJljYXFxdMmzYNOTk5fKwikUKxIBOpzNGjR+Hn51fvN/GgoCAAwHfffSdDKiJqSJu/UxcRWbtw4QJ8fHzqtde11T1w4momkwkmk8ny3mg0tkxAIrKJM2QilamoqIBOp6vX7uLiYum3JS0tDXq93vIyGAwtmpOIrLEgE6mMq6ur1Uy3TmVlpaXfluTkZJSUlFhePNZM1Lq4y5pIZXx8fFBYWP8BJRcuXAAA+Pr62lxOp9PZnFkTUevgDJlIZW655Rbk5ubWOwa8f/9+Sz8RKQ8LMpHKTJgwAbW1tVizZo2lzWQyIT09HcHBwTw2TKRQ3GVNpDLBwcGYOHEikpOT8dtvv6F///5Yv349zp07h7Vr18odj4jsYEEmUqENGzZg3rx52LhxI/78808MGTIEn3zyCUaOHCl3NCKyg7fOJJJBS946s7nw1plEzYO3ziQiImpDWJCJiIgUgAWZiIhIAViQiYiIFIAFmYiISAFYkImIiBSABZmIiEgBWJCJiIgUgHfqIiKb6u4ZdPVDKojo2tR9hxq6DxcLMhHZVFxcDAB8GAVRMyktLYVer7fbz4JMRDZ17twZAPDrr79K/iOiVEajEQaDAfn5+W3y1p/ML6/mzC+EQGlpqd1nkddpdEFuC/feJaLm4+Dw31NM9Hp9m/wHtY6npyfzy4j5/6sxv9TypC4iIiIFYEEmIiJSABZkIrJJp9MhJSUFOp1O7ihNwvzyYv5r1+jnIRMREVHL4QyZiIhIAViQiYiIFIAFmYiISAFYkImIiBSABZmonXjrrbfQp08fuLi4IDg4GAcOHJAc/8EHH2DgwIFwcXHB4MGDsXPnTqt+IQTmz58PHx8fuLq6Ijw8HKdPn1ZE/n/9618YMWIEOnXqhE6dOiE8PLze+KlTp0Kj0Vi9IiMjFZF/3bp19bK5uLhYjVHy9h81alS9/BqNBmPHjrWMac3t/9VXXyE6Ohq+vr7QaDTYsWNHg8tkZ2fj1ltvhU6nQ//+/bFu3bp6Y671O9UgQUSqt3nzZuHs7Czeeecdcfz4cfHwww+Ljh07iqKiIpvjv/32W6HVasXLL78sTpw4IebOnSucnJzEjz/+aBmzePFiodfrxY4dO8T3338vYmJiRN++fUVFRYXs+e+77z7x1ltviaNHj4qTJ0+KqVOnCr1eLwoKCixjEhISRGRkpLhw4YLl9ccffzR79qbkT09PF56enlbZLl68aDVGydu/uLjYKvuxY8eEVqsV6enpljGtuf137twp/vnPf4pt27YJAGL79u2S43/55Rfh5uYmnnzySXHixAnx5ptvCq1WKz777DPLmGvdJo3BgkzUDgQFBYnHH3/c8r62tlb4+vqKtLQ0m+MnTZokxo4da9UWHBwsZsyYIYQQwmw2i+7du4tXXnnF0n/58mWh0+nEe++9J3v+q9XU1IgOHTqI9evXW9oSEhLEuHHjmjuqTdeaPz09Xej1ervra2vbf/ny5aJDhw6irKzM0taa2/+vGlOQn3nmGTFo0CCrttjYWBEREWF5f73bxBbusiZSuaqqKhw+fBjh4eGWNgcHB4SHhyMnJ8fmMjk5OVbjASAiIsIy/uzZs7h48aLVGL1ej+DgYLvrbM38VysvL0d1dbXlgRl1srOz0bVrV/j7++PRRx+1POGqOTU1f1lZGXr37g2DwYBx48bh+PHjlr62tv3Xrl2LuLg4uLu7W7W3xvZvioZ+/ptjm9jCgkykcr///jtqa2vRrVs3q/Zu3brh4sWLNpe5ePGi5Pi6/17LOpuqKfmv9uyzz8LX19fqH9DIyEhs2LABn3/+OZYsWYIvv/wSY8aMQW1trez5/f398c477yAzMxPvvvsuzGYzQkNDUVBQAKBtbf8DBw7g2LFjmD59ulV7a23/prD38280GlFRUdEsP5O28PGLRKRqixcvxubNm5GdnW11YlRcXJzl/wcPHowhQ4agX79+yM7Oxt133y1HVIuQkBCEhIRY3oeGhuLGG2/E6tWrsXDhQhmTXbu1a9di8ODBCAoKsmpX8vaXC2fIRCrn7e0NrVaLoqIiq/aioiJ0797d5jLdu3eXHF/332tZZ1M1JX+dpUuXYvHixdi9ezeGDBkiOfaGG26At7c3fv755+vO/FfXk7+Ok5MThg4dasnWVrb/lStXsHnzZkybNq3Bz2mp7d8U9n7+PT094erq2ix/p7awIBOpnLOzMwIDA/H5559b2sxmMz7//HOrWdhfhYSEWI0HgD179ljG9+3bF927d7caYzQasX//frvrbM38APDyyy9j4cKF+Oyzz3Dbbbc1+DkFBQUoLi6Gj49Ps+Su09T8f1VbW4sff/zRkq0tbH/gv5fOmUwmTJkypcHPaant3xQN/fw3x9+pTU0+HYyI2ozNmzcLnU4n1q1bJ06cOCESExNFx44dLZfSPPDAA+K5556zjP/222+Fo6OjWLp0qTh58qRISUmxedlTx44dRWZmpvjhhx/EuHHjWvSym2vJv3jxYuHs7Cy2bt1qdVlNaWmpEEKI0tJS8dRTT4mcnBxx9uxZkZWVJW699VYxYMAAUVlZKXv+1NRUsWvXLnHmzBlx+PBhERcXJ1xcXMTx48et/oxK3f51hg8fLmJjY+u1t/b2Ly0tFUePHhVHjx4VAMSyZcvE0aNHRV5enhBCiOeee0488MADlvF1lz09/fTT4uTJk+Ktt96yedmT1DZpChZkonbizTffFL169RLOzs4iKChI7Nu3z9IXFhYmEhISrMa///77ws/PTzg7O4tBgwaJ//znP1b9ZrNZzJs3T3Tr1k3odDpx9913i59++kkR+Xv37i0A1HulpKQIIYQoLy8Xo0ePFl26dBFOTk6id+/e4uGHH76uf0ybM//s2bMtY7t16yaioqLEkSNHrNan5O0vhBCnTp0SAMTu3bvrrau1t/8XX3xh8+ehLnNCQoIICwurt8wtt9winJ2dxQ033GB1DXUdqW3SFHz8IhERkQLwGDIREZECsCATEREpAAsyERGRArAgExERKQALMhERkQKwIBMRESkACzIREZECsCATEREpAAsyERGRArAgExERKQALMhERkQKwIBMRESnA/wcrF5rtIERG1QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 600x700 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# check the prediction on same image as the one used before training\n",
    "logits = model(img)\n",
    "\n",
    "# Predict the class from the network output\n",
    "prediction = tf.nn.softmax(logits).numpy()\n",
    "\n",
    "view_classification(img[0], prediction[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that model is able to correctly predict the digit as 3 after training while before training it predicting all digits with almost equal probability i.e. it was randomly predicting the digit."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
